An investigation of feedforward neural networks with respect to the detection of spurious patterns

This thesis investigates feedforward neural networks in the context of classi cation tasks with respect to the detection of patterns that do not belong to the same categories of patterns used to train the network. This refers to the problem of the detection and/or rejection of spurious or novel patterns. In particular, the multilayer perceptron network (MLP) trained with the backpropagation algorithm is examined in this respect and di erent strategies for improving its performance in the detection of spurious patterns are considered. The problem is investigated from di erent points of view that vary from the modi cation of the multilayer perceptron network with di erent con gurations that make it more intrinsically able to detect spurious information, to the introduction of novel auxiliary mechanisms which, when integrated with the MLP network, can provide an overall enhancement in the system's rejection capabilities. These di erent network con gurations are examined with respect to the characteristics of the decision regions constructed by the networks in 2-D classi cation problems, and the implications of these constructions for general pattern rejection are discussed. The technique of inversion in multilayer networks through gradient descent is used to observe the degree of visual correlation between the input patterns recognised as valid by the networks and training class prototypes. Practical experiments on the classi cation of handwritten characters are employed as a test environment for the di erent approaches described. Radial basis function networks (RBFs) are also examined in the same context and an experimental comparison is made between RBFs and the di erent MLP con gurations studied. i Publications Arising From This Work Vasconcelos, G.C., Fairhurst, M.C., and Bisset, D.L. (1995). E cient detection of spurious inputs for improving the robustness of MLP networks in practical applications. Neural Computing & Applications 3(4), 202-212, Springer-Verlag. Vasconcelos, G.C., Fairhurst, M.C., and Bisset, D.L. (1995). Investigating feedforward neural networks with respect to the rejection of spurious patterns. Pattern Recognition Letters 16 (2), 207-212. Vasconcelos, G.C., Fairhurst, M.C., and Bisset, D.L. (1994). Recognizing novelty in classi cation tasks. NIPS'94 Workshop on Novelty Detection and Adaptive Systems Monitoring, Vail CO, USA. Vasconcelos, G.C., Fairhurst, M.C., Bisset, D.L. (1994). Reliability of multilayer perceptron networks for spurious pattern rejection. Proc. 1st Brazilian Symposium on Neural Networks, 29-34, Caxambu MG, Brazil. Vasconcelos, G.C., Fairhurst, M.C., and Bisset, D.L. (1993). Enhanced reliability of multilayer perceptron networks through controlled pattern rejection. Electronics Letters 29 (3), 261-263. Vasconcelos, G.C., Fairhurst, M.C., and Bisset, D.L. (1993). The guard unit approach for rejecting patterns from untrained classes. Proc. 1993 World Congress on Neural Networks (WCNN'93), IV 256-259, Portland OR, USA. Vasconcelos, G.C., Fairhurst, M.C., Bisset, D.L. (1993). Investigating the recognition of false patterns in backpropagation networks. Proc. 3rd IEE International Conference on Arti cial Neural Networks. Brighton, U.K.. ii